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I. Preliminaries 

Let {Pfj \ f3 €z F"^™} be a 1 -dimensional exponential family 
given in a canonical parameterization, 
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where Z is the partition function Z{f3) = J exp(Px) dQx, 
and r*^™ := {f3 \ Z{(3) < oo} is the canonical parameter 
space. We let /3sup = sup{/3 | /3 e T'^™}, and /3inf likewise. 

The elements of the exponential family are also 
parametrized by their mean value /x. We write jip for the mean 
value corresponding to the canonical parameter (3 and (3^ for 
the canonical parameter corresponding to the mean value fi. 
For any x the maximum likelihood distribution is Pp^. The 
Shtarkov integral S is defined as 
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The variance function V is the function that maps /i G 
M into the variance of P^. The Fisher information of an 
exponential family in its canonical parametrization is = 
V (/i/3) and the Fisher information of the exponential family 
in its mean value parametrization is = {V {n))^^ . The 
Jeffreys integral J is defined as 


S = 


More on Fisher information can be found in [2]. 

As first established by [3], if the parameter space is re- 
stricted to a compact subset of the interior of the parameter 
space with non-empty interior (called an ineccsi set in [2]), 
then the minimax regret is finite and equal to the logarithm of 
the Shtarkov integral, which in turn is equal to 
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logJ + o(l). (4) 
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It thus becomes quite relevant to investigate whether the same 
thing still holds if the parameter spaces are not restricted to an 
ineccsi set. Whether or not this is so is discussed at length and 
posed as an open problem in [2, Chapter 11, Section 11.1]. 

II. Results 

Theorem 1: For a 1 -dimensional left-truncated exponential 
family, the following statements are all equivalent: 

1) The Shtarkov integral is finite. 

2) The minimax individual-sequence regret is finite. 

3) The minimax expected redundancy is finite. 


4) The exponential family has a dominating distribu- 
tion Qdom in terms of information divergence, i.e. 

SUp^gr™ ^(-P/sllQdom) < 00. 

5) There is distribution Pp with /3 e F"^™ that dominates the 
exponential family in terms of information divergence. 

6) The information channel (3 ^ Pp has finite capacity. 

7) There exists /3o e F'^™ such that 

lim D((3o\\(3)<oo or lim DipWo) < 00. 

Most of the equivalences between (l)-(6) are quite straight- 
forward. The surprising part is the fact that statements (l)-(6) 
are also equivalent to (7). 

Theorem 2: Let (Fq™,(5) represent a left-truncated expo- 
nential family. If the Shtarkov integral is infinite, then the 
Jeffreys integral is infinite. 

The converse does not hold in general. 

Theorem 3: Let (Fq™, Q) represent a left-truncated expo- 
nential family such that /3sup — and Q admits a density q 
either with respect to Lebesgue measure or counting measure. 
If q{x) — 0{l/x^^") for some a > 0, then the Jeffreys 
integral Jp li^Y^^ ^7 is finite. 

In most cases finite Shtarkov implies finite Jeffreys. 

Theorem 4: Let Q be a measure on the real fine with 
support /. Assume that /ijnf is the left end point of /. If Q 
has density / (x) — {x — fJ.iniy~^ 9 {x) in an interval just to 
the right of a where g is an analytic function and g (/iinf ) > 
then the left end of the interval / gives a finite contribution 
to Jeffrey's integral if and only if Q has a point mass in a. 

If F is a Cauchy distributed random variable then X = 
exp (Y) has density 
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A probability measure Q is defined as a 1/2 and 1/2 mixture 
of a point mass in and an exponentiated Cauchy distribution. 
The exponential family based on Q has redundancy upper 
bounded by 1 bit but the Jeffreys integral is infinite. 

For exponential families in more dimensions the analysis 
becomes more more involved and one may even have expo- 
nential families with finite redundancy and infinite regret. 
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